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ABSTRACT 

There is widespread interest among discipline-based science education researchers to situate their research in the 
existing scholarly literature base. Unfortunately, traditional approaches to conducting a thorough literature review 
are unduly hindered in astronomy education research as the venues in which scholarship is reported are fragmented 
and widely dispersed across journals of varying disciplines. The international STudies of Astronomy education 
Research (iSTAR) online repository is the result of a concerted international community effort to collect and 
categorize existing research from peer-reviewed journal articles, dissertations/theses, and grey literature. In a “first 
light” survey of over 300 U.S. dissertations, we find: (i) work in AER dates back to 1923; (ii) the number of extant 
dissertations is far greater than anticipated; (Hi) research methods definitions have evolved; and (iv) most work has 
studied participants’ broad knowledge rather than specific learning targets. The surprisingly wide breadth of rarely 
cited research motivates us to collect more AER from across international and cross-disciplinaiy sources. 
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rogress in astronomy education research—heretofore referred to simply as AER—requires that we first 
understand the scope of what is known about teaching and learning astronomy content across diverse 
settings, and what is not. Unfortunately, AER is a broad scientific discipline in which the existing 
landscape is particularly difficult to comprehensively grasp. Unlike the field of traditional astronomy in which research 
is conducted nearly exclusively by individuals who hold the exclusive identities of “professional astronomers,” AER 
is conducted by individuals representing a wide variety of academic settings and affiliations, including professional 
astronomers, astronomy education researchers, planetarium operators, cognitive scientists, psychologists, 
anthropologists, science educators, science education researchers, teacher educators, policy makers, museum 
educators, curriculum developers, outreach enthusiasts, and publishers (Slater, Slater, Heyer, & Bailey, 2015). These 
individuals most often present findings in research journals tightly related to their own professional fields, hindering 
knowledge transfer amongst communities of researchers and practitioners. Absent the ability to accurately describe 
the existing research landscape we often swirl in the vortex of our preconceptions, stymied in our efforts because we 
do not have a grasp on what we do and do not know, about the ways that humans currently and might possibly come 
to learn astronomical concepts. 


Isolation caused by disciplinary silos is magnified by other factors that drive AER toward domination by a few voices, 
from a few research settings, using a limited set of theoretical frameworks and methodologies. Works published as 
dissertations and thesis, in the pre-digital era, from lesser known journals coming from non-Western countries, or that 
are hidden behind paywalls are disadvantaged in their ability to influence the AER research agenda. Researchers from 
smaller institutions and from developing nations experience the greatest barriers to access these community resources 
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and, thus are disproportionately affected by the lack of access to prior research and by difficulties in disseminating 
their work. This situation unnecessarily diminishes scholars’ opportunities to collaborate and to influence the 
international research discussion. We would further assert that even dominant researchers are hindered from engaging 
in fruitful collaborations and research by the existing barriers to accessing the widely distributed, difficult to acquire 
literature relevant to the field. 

The lack of diverse collaborations and cross-pollinations within the field is not a merely academic problem. Rather, it 
may help to explain much of what we have seen in the existing Western literature base. That is to say, while we have 
experienced a large production of research literature, we continue to struggle to sufficiently improve student learning 
in astronomy. For instance, when we look at recent reports of students’ conception of the Sun-Earth-Moon system 
(Slater, 2014), we see little improvement from reports that date back to the early years of AER (Ralya & Ralya, 1940). 
Despite what appears to be an excess of research on the topic (Lelliott & Rollnick, 2010), Western students still cannot 
explain the cause of the seasons. While there are a few possible explanations for these results, one possible explanation 
may be that the dominant perspectives in the field have converged on an AER canon that is a self-validating reduction. 
If so, one possible solution may be to look beyond the most cited literature, and to mine for new insight in the vast 
body of work provided our colleagues, who may have considered these learning challenges from uncommon 
perspectives, using unfamiliar instructional techniques, in diverse cultural settings, employing atypical methodologies. 

Given this potential, we were encouraged to create a more efficient pathway for international researchers to build from 
a wider diversity of AER scholarship. After determining that there is indeed evidence that, even in research university 
settings, there is a great deal of difficulty in obtaining existing research publications, we constructed the international 
STudies of Astronomy education Research (/STAR) research online repository. /STAR is intended to function as a 
research instrument for the AER community, eventually housing the majority of AER publications produced across 
the international setting. This paper reports on the instrument’s “first light” functionality as tested on a small sample 
of papers. A term borrowed from astronomy, “first light” is the first image taken with a new research instrument, and 
while never the most important or most meaningful image that will ever be taken, it is usually of a target that is well 
known, and perhaps of sentimental value to the community. In this case, iSTAR’s first light image is of the body of 
existing AER dissertations and theses produced in the United States, a sample that we initially believed would number 
approximately 50 works. 

This paper describes our process of developing the /STAR database, and provides a glimpse of what the database may 
be capable of via a first light look at over 300 AER doctoral dissertations and graduate theses from the United States 
related to astronomy education. Future steps for the /STAR project and opportunities to contribute to the project are 
described in the discussion section. 


BACKGROUND AND CONTEXT 

Prior to constructing /STAR a review was conducted of existing literature reviews in AER. The intention was to 
determine the extent to which these reviews included similar works, and the extent to which the cited works were 
easily accessed online and through the libraries of research institutions. For the purposes of the efforts described here, 
our survey of the pre-existing reviews is constrained to six published reviews. A summary of the selected reviews is 
shown in Table 1. 

Wall (1973) was the first to publish an extensive summary of research related to astronomy education in a peer- 
reviewed journal. The works he summarized were primarily theses and dissertations created by graduate students 
completing a University degree. It should be noted that in the narrative of his paper, Wall states that 54 of the 58 
reviewed works were dissertations or theses. In the “References” section of his paper, only 36 works are clearly so 
identified. Wall separated the included works by the nature of each study’s participants (e.g., elementary, secondary, 
and college-level, teachers) and by the type of study (e.g., status studies, achievement gains, and curriculum 
summaries). His analysis was not restricted to a content domain of astronomy nor was it restricted to a geographic 
region as he included research reports he could find outside of the United States’ borders. Wall’s report was critical 
to the field of astronomy education research at that time in that it was the first of its kind to attempt to collect works, 
describe them, then propose recommended future directions to the field. 
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Table 1. Summary of Peer-Reviewed Literature Reviews in Astronomy Education 


Paper 

Year 

Content Scope 

Geographic 

Scope 

Document Types 

Analysis Method 

Number of 
Papers Included 

Wall 

1973 

All 

International 

Theses & 

Dissertations 

Annotated 

Bibliography 

58 

Bailey 
& Slater 

2003 

All 

International 

Peer-reviewed Work & 
Sadler’s Dissertation 

Annotated 

Bibliography 

<100 

Fraknoi 

2007 

All 

International 

Peer-reviewed Works 
(AER Journal Papers) 

Descriptive 

Statistics 

Approximately 

100 

Brazell 

2009 

Planetarium 

U.S. 

Peer-reviewed, 

Dissertations 

Quantitative Meta- 
Analysis 

19 

Lelliott 

2010 

Broad Astronomy 
Content 

International 

Peer-reviewed 

Research Synthesis 

103 

Bretones 

2011 

All 

International 

IAU Proceedings 

Descriptive 

Statistics 

283 


Wall’s work provided important context for the /STAR project. First, Wall’s sample indicated that given a steady rate 
of production, there might be as many as 100 dissertations and theses available for initially populating the repository, 
allowing the repository to be tested for functionality. However, of the theses and dissertations cited in Wall’s review, 
the majority were not readily accessible through the normal channels at three U.S. research institutions. We consider 
this evidence that an online repository would greatly enhance access to these types of works. 

Thirty years later, Bailey and Slater (2003) reviewed more than 100 studies referencing astronomy education topics. 
They included peer-reviewed articles, books, and web-based materials in their analysis. Including all content areas, 
the researchers organized the astronomy education research literature they could find into two broad categories: (i) 
student learning/understanding and (ii) instructional methods. The student learning category was then subdivided into 
a variety of conceptual learning targets including lunar phases, shape of the Earth, diurnal motion, cosmology, and 
astrobiology. Instructional methods were subdivided into personalized instruction, teaching conceptual understanding, 
collaborative learning, specific curriculum interventions, and planetarium learning. With regards to geographic scope, 
Bailey and Slater (2003) included all works located and related to astronomy education research largely within and 
somewhat beyond the borders of the United States. This is somewhat similar to work done in Australia by Broadfoot 
and Ginns (2005). 

Like Wall did decades earlier, Bailey and Slater’s goal was to create a landscape portrait of the field and make 
recommendations for further research efforts. However, the included works are exclusively from journals published 
in Western hemisphere countries. The two projects had surprisingly little overlap with only three of the works 
reviewed by Wall also appearing in the Bailey and Slater review. According to the authors, at the time of the review, 
works from other contexts and works cited by Wall were prohibitively difficult to find, despite efforts to obtain the 
remaining works through typical channels at a research-intensive university. 

In 2007, Fraknoi and Wolff reviewed the progress and presented the descriptive statistics of articles appearing in the 
Astronomy Education Review journal. Their goal was not to review the entire field of AER, but to review articles 
appearing in the then relatively new journal to evaluate the journal’s impact, its readership, and propose a pathway for 
the future of that peer-reviewed journal. At that time, there were approximately 100 peer-reviewed articles from the 
journal, which encompassed an international scope. Concluding that the Astronomy Education Review journal was 
having an important impact on the field of AER, the authors highlighted the frequency of articles from a specific 
institutions and subdivided the journal articles into themes including: teaching astronomy for K-12, use of technology 
in astronomy education, public outreach and education, and undergraduate and graduate-level education. This review 
is similar to that published by Bretones, Jafelice, and Horvath (2016). 

The Fraknoi and Wolff review highlights the value of Internet-available journal like th a Astronomy Education Review, 
in that it provided a locus for astronomy education researchers across international contexts to interact across open 
access to relevant works. The online, open access nature of the journal’s publications supports research and practice 
across all settings with Internet access, serves as an important model for the newly developed /STAR platform 
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(Fraknoi, 2014). Two years later, Brazell (2009) reported on a meta-analysis of peer-reviewed articles and 
dissertations related to the efficacy of learning in the planetarium. Working from 19 United States publications Brazell 
quantitatively synthesized these planetarium studies by looking at effect sizes for achievement, attitude, grade level, 
subject matter, treatment type, control of instructor effect, group assignment, frequency of visits, and period (1960- 
1984 or 1985-2008). A comparison of the review completed by Brazell and with the Wall review (1973), which 
includes a large number of planetarium works indicated that in overlapping years, the papers differed in the 
publications captured for analysis. Rather than serving as evidence of poor scholarship, this serves as evidence of the 
difficulty inherent in locating relevant works, even for scholars who are thorough in their research. 

A 2010 literature review by Lelliott and Rollnick included 103 peer-reviewed journal articles focused on studies 
occurring within the primary and secondary classrooms, museums, and/or among pre-and in-service teachers, as 
opposed to considering papers on postsecondary courses. They used peer-reviewed print based journal articles that 
included empirical, conceptual, and review papers, but purposefully excluded historical analyses, practitioner, and 
opinion pieces. They further synthesized all the works focusing on what they identified as the “big ideas” in astronomy. 
A comparison of this review with Bailey and Slater (2003) indicated that each paper captured works that were 
overlooked by the other review, when considering stated review parameters. Bailey and Slater were more likely to 
include works from lesser known U.S. sources, whereas Lelliott and Rollnick ventured into journals not captured in 
the Bailey and Slater review (e.g., Learning and Instruction Journal). This suggest that the dispersal of works in a 
wide variety of publications has the potential to expand our conception of the AER landscape. 

Encompassing a much larger range of works, Bretones and Megid Neto (2011) looked for trends and synthesized 
characteristics across 283 published proceedings papers on astronomy education research presented at the 
International Astronomical Union meetings from 1988 to 2006. These proceedings papers are significantly longer and 
more detailed than typical abridged abstracts or poster presentations, which were not included in the analysis. The 
purpose of their study was to give recommendations for needed further research based on the statistics used to describe 
the field, using an international lens. Most relevant to the /STAR project, none of the works included in this review 
were included in any of the other reviews. Bretones and Megid Neto reported that over 60% of the published 
proceedings were contributed by authors from outside of the United States. In comparison, the vast majority of works 
included in the previous reviews were contributed by U.S. authors. This provides warrant for the creation of pathways 
that highlight, and make accessible, non-U.S. research. 

The existing reviews of astronomy education research described above vary greatly in their analysis methods, types 
of articles, and the number of articles included in their synthesis. Somewhat unexpectedly the reviews were 
universally non-exhaustive, as evidenced both by a significant lack of common overlap among the papers’ reference 
lists, even when considering that the works differ in their intentions. Perhaps this is due to astronomy education 
researchers themselves having more interest in conducting limited and more focused reviews that serve their interests. 
For example, Adams and Slater (2000) published a highly constrained, but in-depth, review only considering what 
was known from astronomy education research only on those few topics listed in the United States’ National Science 
Education Standards which would not normally be categorized as a comprehensive literature review. More likely, 
we attribute the lack of all-encompassing and synthetic reviews to the fact that librarians and the Internet have only 
recently made the task of finding work across the field significantly easier than it was in previous decades, although 
the increased access still does not extend to pre-digital works, works published in other nations, or works behind 
paywalls. This further adds warrant to the notion that a new effort to create a contemporary comprehensive list—if 
not a central online repository—would be valued and highly used by scholars in the field. 

METHOD 

Responding to the astronomy education community’s need for a more useful astronomy education research literature 
collection, the /STAR project began by selecting a test sample of papers and by determining a method for constructing 
the repository in such a way as to maximize its effectiveness for researchers. 
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Sample Selection 

For the initial /STAR database survey, dissertations and graduate theses were chosen as the test sample. Many of these 
works are in danger of being lost as many, particularly those written prior to 2000, were never published beyond their 
initial committee review and printing. Today these works are often not considered to be peer-reviewed, but in earlier 
years more weight was given to the extraordinary amount of time and energy advisors and committee members 
contributed to the quality of the final publication. As an additional benefit, collecting the body of AER dissertations 
could give us an idea of the astronomy education “family tree” by taking note of who conducted degree work in 
astronomy education and who sat on those candidates’ committees. Most importantly, the number of expected 
dissertations was fairly small. Given our collective experience in the field, we expected to find between 50 and 100 
such papers produced in the United States. 

While collecting community feedback on the repository structure sample collection began by scouring online library 
databases such as ProQuest, ERIC , university library repositories, Google Scholar and WorldCat using keywords such 
as “astronomy education”, “constellations”, “lunar phases”, “seasons”, “planetarium”, and “cosmology”. The citations 
list of all collected works were searched for references to additional papers that may not have appeared in any of the 
indexing systems. This system returned more than 400 dissertations and graduate theses. Citations for over 100 
additional dissertations and theses were found, but we were unable to locate either the hardcopy paper or even the 
abstract. It is possible that more exist. 

The content of each work was reviewed. If a work was deemed to be unrelated to astronomy education it was discarded. 
After that process, more than 300 dissertations and theses related to astronomy education remained. For the purposes 
of this “first light” study, only those works for which we had both a complete abstract and main narrative body were 
included in the analysis. This reduced the number of scholarly works in the initial sample to 252. 

/STAR Categorization Scheme 

/STAR was designed with the intention to store publications in a manner most useful to researchers. As a first step, 
possible types of research reports were categorized into three categories: peer-reviewed articles, dissertation and 
theses, and grey literature. Grey literature is a widely-used term referring to professional conference contributions, 
transcripts or videos of invited keynote presentations and plenary talks, meeting proceedings, books and book chapters 
on astronomy teaching excellence, and Internet blog posts, as well as seminar and workshop presentations {viz.. Slater, 
2015; 2016). Even though not often rigorously peer-reviewed, it is our belief that highly insightful and valuable 
information for astronomy education researchers, when explicitly labeled as such, is also contained in the domain of 
grey literature. As an example of grey literature scholarship, the Appendix provided lists some of the grey literature 
scholarship leading up to publication of this article. 

The following categories were developed to create an initial scheme that would be most helpful to researchers in the 
field, and facilitate searches for relevant works pertaining to any research topic of choice. Those categories were sent 
out to 15 international experts in astronomy education for review and comment. Definitions were then written for each 
of the header and sub-categories to set the boundaries for how items would be classified. This new draft was also sent 
out to 15 international experts in the astronomy education field for comments and suggestions. The experts came from 
a variety of countries including Australia, Brazil, Japan, Denmark, Portugal, Spain, South Africa and the United States. 
This panel of experts helped redefine and edit the categories. Table 2 shows the developed /STAR categorization 
scheme and their definitions. 


Copyright by author(s); CC-BY 


129 


The Clute Institute 





Journal of Astronomy & Earth Sciences Education - December 2016 


I plume 3, Number 2 


Table 2. Categories for Reviewing Astronomy Education Research Reports 


Category 

Definition 


Document Source 


Peer-reviewed 

Published academic or scientific work judged as valuable, timely, and rigorous by other 
scholars in the field 

Dissertation or Thesis 

A paper that is used to fulfill the requirement of a higher educational degree, juried by 
university faculty 

Research presented at conferences, on websites, in newsletters produced by organizations 

Grey literature 

or societies hut not through commercial channels, e.g., pre-prints, conference 
proceedings, statistical reports, preliminary progress reports 


Type of Resource 


Literature review 

A comprehensive comparison and contrast paper which critically summarizes or 
synthesizes the current knowledge across an academic landscape 

Empirical research 

A scientific study conducted using data collected via observation or experiment 

Theoretical research 

A study describing a pre-existing, redefined, or new theory that makes a priori predictions 
about proposed experiments 

Curriculum or program evaluation 

An assessment on the effectiveness of an education program or curriculum design 

Curriculum description or report 

A paper describing and documenting an educational program or curriculum design 

Position paper or editorial 

A non-refereed opinion-oriented publication 

Historical 

A paper describing past events and its influence today 

Bibliography or resource guide 

A summary of the literature or annotated guide to existing literature or educational 
resources 


Empirical Methodology 


Quantitative 

Research designed to quantify an educational state, or change of state, by transforming 
observations into numerical data that can be statistically analyzed 

Research designed to explore underlying causal mechanisms occurring and meaning- 

Qualitative & Interpretive 

making participants experience interpreted without extensive use of traditional statistical 
analysis 

Mixed-methods 

A study using both qualitative and quantitative methods to triangulate an evidence based 
conclusion from multiple philosophical perspectives and viewpoints 


Learning Environment 


Formal 

Context is school-like classroom setting, led by trained instructors, with specified and 
assessed learning targets 

Informal 

A study conducted outside of a classroom setting (e.g., after-school programs, museums, 
star parties, field trips, etc.) 


Planetarium 

Any study incorporating the use of a physical planetarium 

Museum 

Science learning center or facility that preserves historical artifacts 

Amateur Groups or Activities 

A study involving non-professional (e.g., star parties, variable star data collected via 
amateur astronomers, etc.) 

Extracurricular/Camps/After- 

school/Scouting 

Non-school related educational activities that are rarely graded 

Home school 

Students who do not attend a traditional brick-and-mortar school building 

Online, virtual 

Students who do not attend a traditional brick-and-mortar school building and take 
synchronous or asynchronous classes via an Internet connection 

Citizen science 

Individuals who are not paid professional scientists who are contributing to scientific data 
collection or analysis 

Research facility 

National laboratory or scientific research institution that is not a college, university, or 
museum 

(Table 2 continued on next page) 
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(Table 2 continued) 

Category 

Definition 

Study Participants 

Early childhood students 

Younger than 5 years of age 

Elementary students 

Students ages 5-11 years of age 

Middle & secondary school students 

Students ages 11-18 years of age 

College students 

Students enrolled in an undergraduate course 

Second language learners 

Students enrolled in a course or school not taught in their first language learned 

Graduate students 

Students pursuing higher education degrees such as a doctorate or masters 

Pre-service teachers 

College students pursuing a career as a teacher 

In-service Teachers 

Teachers currently educating students ages 3-18 years old 

College faculty 

Participants teaching at a higher education institution 

Multi-aged groups 

Family or community groups with varying ages 

Adult learners 

Retirees or third-age learners (e.g., Elderhostel) 

Construct 

Content knowledge 

Facts, concepts, theories, etc. 

Affective variables 

Belief, identity, motivation, attitudes, etc. 

Cognitive processes 

Attention, memory, working memory, judgment, reasoning, etc. 

Quantitative reasoning 

Ability to read and understand mathematical problems 

Spatial reasoning 

The capacity to visualize and manipulate mental images 

Research & methods assessment 

The research methodologies used in empirical and evaluation studies 

General teaching strategies 

The didactic, pedagogical, praxis, and educational engagement process 

Scientific inquiry 

The processes involved in developing scientific knowledge 

Nature of Science 

The understanding of science as a way of knowing 

Scientific Content Focus Area 


General/Broad Knowledge of Astronomy Content 


Stars 

Sun-Earth-Moon 

Solar System 

Scale & Structure 

Cosmology 

Atoms & Light 

History of Astronomy 

Astrobiology & Exoplanets 

Galaxies 

Gravity 

Cultural Astronomy 
Other _ 

Demographic Focus _ 

Multicultural or indigenous 
Gender & sex differences 
Disability 
Gifted 

Low socio-economic status 
Other _ 

Language of Publication 

Paper written in English 
Abstract only in English 
Non-English publication 

Location Study Conducted 

USA 

Other 
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Graphical User Interface Design 

To meet the goal of making scholarly astronomy education research works as easy to survey and analyze as possible, 
an Internet-accessible search form was designed by the authors and their colleagues. The underlying programing 
system was designed by Open Repository in the United Kingdom, who designed the database using an open source 
DSpace protocol. Initial seed funding was provided by the CAPER Center for Astronomy & Physics Education 
Research in the U.S., and by Open Repository who use their resources to support scholarly efforts that increase 
equitable access across international settings. 

The /STAR database search pages are intentionally created to be easy to use for international scholars with limited 
English proficiency. For the /STAR database to withstand the vagaries of unstable funding, astronomy education 
research community members themselves can add files and categorize their own uploads. Each of the works submitted 
to the database can be vetted through a moderated “workflow” process, where approved administrators can either 
accept/rcject the items, or edit the categorizations. There are no restrictions for access. 

FINDINGS 

This surprisingly large initial sample of scholarly work glimpsed through this first light survey of astronomy education 
research dissertations is analyzed using descriptive statistics. Further inferential analyses will be the subject of 
subsequent papers. The tables and plots provided in this section illustrate the power of the /STAR database by 
describing the number of works generated per year, the number of works per institution, dominant type of research 
methods employed, learning environments studied, characteristics of study-participants surveyed, definitions and 
constructs framing the work, and the scientific content domains under study. 

Year of Dissertation or Thesis Completion 

Table 3 provides the frequency count of papers in the United States in the existing /STAR database in five year bins 
over the 89-year span of the database, as of this writing. It is important to note that this dataset is incomplete and most 
certainly does not yet exhaustively represent the full history of the field. In building the database, more recent works 
are often far more accessible via online databases, both in terms of finding the title and author of a paper, and in terms 
of being able to acquire a digital copy of the work. In particular, many known works dating from the 1960s, 1970s 
and 1980s have never been digitized, and are not available through traditional interlibrary loan programs because there 
is only one paper hardcopy. Therefore, any inference from the current dataset that the field of astronomy education 
research has been more productive in the most recent years is unsupported; instead, it stands as witness to the notion 
that much of our research heritage has been heretofore largely unavailable to many scholars. 


Table 3. Distribution of 252 Initially Recovered Graduate Dissertations and Theses across Bands of Time 


Years 

Number of works 

Years 

Number of works 

1925-1929 

1 

1975-1979 

9 

1930-1934 

2 

1980-1984 

12 

1935-1939 

0 

1985-1989 

13 

1940-1944 

1 

1990-1994 

22 

1945-1949 

3 

1995-1999 

25 

1950-1954 

0 

2000-2004 

25 

1955-1959 

2 

2005-2009 

48 

1960-1964 

6 

2010-2014 

61 

1965-1969 

10 

2015 

3 

1970-1974 

9 




Institutions Supporting Astronomy Education Degree Work 

Table 4 provides a frequency count of dissertations and theses per institution for those institutions producing more 
than four degrees, as cataloged in the database as of this writing. In total, 101 post-secondary institutions of higher 
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education (colleges and universities) contributed to this body of work, with ten institutions producing nearly a third 
of all degrees in the dataset. It is worth noting that work produced in these high-production institutions is concentrated 
within relatively short periods of time. For example, the University of Southern California dominated scholarly 
production in astronomy education research in the 1930s and 1940s then suddenly ceased. In the case of Oklahoma 
State University, top production of degrees occurred in the 1990s under the supervision of a group of faculty who 
have since moved to other institutions or have retired. As another example, Montana State University’s productivity 
dramatically increased in the 2010s with the publication of theses resulting from a new Master’s of Science degree in 
Interdisciplinary Science Education program. 

Whereas sometimes particular universities become dominant producers of scholarly work because of its degree 
programs, other universities become dominant producers because of particular individuals. This is particularly true 
in AER, where no formal departments exist, and production is driven by the presence a single scholar at an institution. 
For example, production of astronomy education dissertations and theses at The Ohio State University peaked in the 
2000s, abruptly ending with the departure of Dr. Kathy Trundle, who left that institution to become the department 
chair for Science, Technology, Engineering, and Mathematics (STEM) education at Noith Carolina State University. 
Peak scholarly production at the University of Arizona occurred during the 2000s and sharply declined after the 
departure of Professor Timothy Slater who went on to an endowed professorship at the University of Wyoming. The 
University of Wyoming’s graduate theses and dissertation production skyrocketed starting in 2010. This pattern 
should be expected to continue. For example, while Penn State does not currently appear on the list of top institutional 
producers, we might expect that Penn State will see a dramatic increase in productivity in this area with the arrival of 
planetarium education scholar, Dr. Julia Plummer. 


Table 4. Institutions Producing the Largest Number of Astronomy Education Dissertations and Theses as of 2015 


Institution 

Number of degrees produced 

University of Arizona 

16 

Montana State University 

14 

University of Southern California 

11 

The Ohio State University 

8 

Oklahoma State University 

7 

University of Wyoming 

7 

Columbia University 

6 

Temple University 

6 

University of Iowa 

6 

University of Michigan 

6 


As emphasized in the previous section, this initial “first light” sample is certainly incomplete, but provides a unique 
glimpse into the demographics of astronomy education research conducted in partial fulfillment of graduate degrees 
earned in the United States. 

Dissertation or Graduate Thesis Research Methods 

Within the /STAR database, all complete entries for empirical works entered to date are categorized as quantitative, 
qualitative or mixed-method studies. Whenever possible this classification was determined through the abstract’s 
explicit statements. When the work did not explicitly declare its research method, a determination was made by the 
coder, often by carefully reading the methods section of the work. While quantitative works tend to be relatively easy 
to classify quickly, classification for qualitative and mixed-method works can be more open to interpretation. Prior to 
the 1980s, it was common to characterize research as qualitative if it involved data that began analysis in non- 
numerical form, was later quantified, and statistically analyzed: Today, such a study would be described instead as 
quantitative even if the data was initially non-numeric in form if it is eventually reported in terms of frequency counts. 
The period between 1980 and 2000 is known by many across the discipline-based education research community as 
the time of the “paradigm wars” (Gage, 1989). During that period, community-wide battles for an expansion of the 
accepted research methods and methodologies raged, resulting in a research community in which non-experimental 
and non-quantitative methods were increasingly accepted as legitimate research practices. During the paradigm wars 
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and since, qualitative research tends to be, but is not always defined as, research that has as its focus the contextually 
significant experience of the participants (Erickson, 1986), a perspective which would frequently exclude papers 
designated as qualitative work prior to the paradigm wars. A determination of “mixed methods” could therefore mean 
a variety of things, depending upon the era and the perspective of the author. For the purposes of the coders of the 
initial sample loaded into /STAR, the coding scheme used is largely based on Niglas’ (2000) work. Having said that, 
the categorization of works using this scheme is imprecise and others may occasionally disagree with the self¬ 
categorization completed by the authors themselves. 

Nonetheless, the initial data set in the /STAR database that included both an abstract and a narrative body judged to 
be closely related to astronomy education were categorized as using quantitative (n=105), qualitative (n=58), or mixed 
methods (n=56). More interestingly, the proportion of research methods employed has evolved over time. The data 
were disaggregated into three domains: pre-paradigm wars (1927-1980), paradigm wars (1981-2000), and post¬ 
paradigm wars (2001-2015). During all periods, quantitative research methods dominate and qualitative and mixed 
methods increase as shown in Table 5. 


Table 5. Distribution of Research Methods Used by Research Era 



1927-1980 

1981-2000 

2001-2015 

Quantitative 

76% 

52% 

37% 

Qualitative 

5% 

29% 

32% 

Mixed Methods 

18% 

19% 

31 % 


Data were also examined to see if there are any institutional trends at those institutions that produce the greatest 
number of works. While the numbers are small, there does seem to be a research “preference” at three of the top 
producers. The University of Arizona and The Ohio State University showed a tendency towards qualitative and 
mixed methodologies during their periods of productivity (12 out of 16 works, and 7 out of 8 works, respectively). In 
contrast, Montana State University exhibited an obvious research preference toward quantitative methodologies (10 
out of 14 works). Other frequent producers showed no meaningful tendencies one way or the other. 

Beyond Classroom Research Settings 

Table 6 provides the frequency of works in the database that were specifically conducted outside of classrooms-like 
settings (n=77). While the eventual inclusion of not-immediately-available, non-digitized works from the frenzied 
planetarium-building construction phase of the 1950s and 60s will likely increase the number of categorized works 
conducted in non-formal out-of-school type settings, it is unlikely to shift the general trend: graduate theses and 
dissertations about planetarium learning environments represent the clear majority of works (n=54, 70%). 


Table 6. 77 Dissertation & Graduate Theses Listed by Informal Research Setting 


Research Setting 

Number of works 

Planetarium 

54 

Extracurricular/Camps/After-school 

4 

Online/Virtual 

4 

Museum 

3 

Research Experiences 

2 

Amateur Groups 

2 

Citizen Science 

2 

Indigenous Site 

1 

Music 

1 

Observatory 

1 

Research Facility 

1 

Sacred Mountains 

1 

Star Party 

1 
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Subjects and Participants Studied 

The /STAR database allows users to investigate the characteristics of the study-participants, allowing for multiple 
selections per work. Thirty-one dissertations and theses in the sample did not actually involve human subjects. These 
works performed curriculum studies, historical analyses, and descriptions of planetarium program or facility 
development. The remaining dissertations and theses described characteristics of 236 different participant groups. 
Studies of middle and secondary students (n=71, 30.1%) and college students (n=58, 24.6%) make up more than half 
of the participants involved, as shown in Table 7. 


Table 7. Frequency of Participant Group Categories in 236 Dissertations and Graduate Theses 


Participants 

Works 

Percentage of all works 

Middle/High Sch. Students 

71 

30.1% 

College Students 

58 

24.6% 

Elementary Sch. Students 

35 

14.8% 

Practicing Teachers 

29 

12.3% 

Multi-aged Groups 

13 

5.5% 

Future Teachers 

11 

4.7% 

College Faculty 

6 

2.5% 

Adult Learners 

4 

1.7% 

Graduate Students 

2 

0.8% 

Scientists 

2 

0.8% 

Families 

1 

0.4% 

Pre-primary Students 

1 

0.4% 

Staff 

1 

0.4% 

Indigenous Peoples 

1 

0.4% 

Parents 

1 

0.4% 


With regard to the specific demographic groups in which education researchers are interested, these evolve over time. 
Data were disaggregated to determine any trends that could be observed. Percentages were calculated to determine 
the types of subjects (i.e., adults, older children, and younger children) studied in each of the research eras described 
in the previous section. The number of studies categorized for the early eras is relatively small (n=29), which may 
influence the trend observed. However, it appears that the number of studies involving young children have declined, 
while those involving adults, particularly in-service and pre-service teachers, have increased. Studies involving older 
school children in middle and high school have remained steady, and are shown in Table 8. It is possible that the 
observed trend reflects a general shift in the focus in astronomy education. Alternatively, this trend may be a natural 
research community response to the development of more stringent Human Subjects Protections in the mid-1970s, 
(Slater, Slater, Heyer, & Bailey, 2015, p. 118; H.H.S., n.d., para. 6) and its increasing impact in the social sciences. It 
is possible that the growing difficulty in conducting research involving minor-aged school children is a barrier to such 
studies being conducted at the graduate school level. A similar analysis involving works completed by faculty rather 
than graduate students may provide additional insight. 


Table 8. Distribution of Participant Groups Studied by Research Era 


Age Group 

Research Era 

Age Group 

Research Era 

1927-1980 

1927-1980 

Adults 

40% 

Adults 

40% 

Older Children 

40% 

Older Children 

40% 


Research Constructs Framing the Studies 

Traditionally, the “research construct” is the proposed attribute that often cannot be measured directly, but can 
potentially be assessed using a number of proxy indicators or assessments (Cronbach & Meehl, 1955; Messick, 1995). 
In astronomy education research, we are often interested in measuring school children’s’ conceptual or content 
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knowledge levels, the depth of understanding of the nature of science, and their attitude or affect toward science. 
When we study adults—teachers, faculty and scientists—we are still interested in these constructs, and we might also 
be interested in their general teaching ability, or their spatial reasoning skills. In the /STAR database, entries are coded 
for the construct(s) studied—a given work may involve more than one construct of interest. The 236 publications that 
could be sensibly categorized for “construct” produced 373 construct codings. It is perhaps unsurprising to most 
scholars that the two most studied constructs in the sample are content knowledge (n=129, 35%) and affect (n=77, 
21%). It should be noted that the “affect” category is a mixture of interest, value, attitude/belief, and identity, as the 
ways in which those terms have been used over the decades is often interwoven, overlapping, and difficult to parse 
out. These are illustrated in Table 9 and in Figure 6. 


Table 9. Frequency of Constructs Studied in 236 Dissertations and Graduate Theses 


Construct 

Number of Codings 

Percentage of Codings 

Content Knowledge 

129 

35% 

Affective Belief/Identity Motivation/Attitude 

77 

21% 

General Teaching Strategies 

70 

19% 

Spatial Reasoning 

26 

7% 

Cognitive Processes 

22 

6% 

Nature of Science 

18 

5% 

Scientific Inquiry 

15 

4% 

Quantitative Reasoning 

7 

2% 

Communication Skills 

2 

1% 

Research Methods/Assessment 

2 

1% 

Technology Use 

4 

1% 


Astronomy Content Knowledge Studied 

Of the 145 works that could be categorized in this way, Table 10 provides the frequency of domains of astronomical 
content knowledge studied. In an effort to achieve some type of coherence, content areas were collapsed whenever 
possible. For instance, if a dissertation was studying students’ understanding of the Fl-R Diagram, it was categorized 
as focusing on the broader category of “stars,” even though that larger category naturally misses some of the nuance 
of the work. Over the 89 years of astronomy education research works sampled for this paper, “General 
Knowledge/Vocabulary” has, by a wide margin, been the most common topic studied (n=52, 36%). The “Sun-Earth- 
Moon” system, which includes the topics of lunar phases and seasons was a common second choice (n=32, 22%). 


Table 10. Frequency of Content Areas Studied in 145 Dissertations and Graduate Theses 


Content 

N umber of Works 

Percentage of Categorized Works 

General Knowledge & Vocabulary 

52 

36% 

S un-Earth-Moon 

32 

22% 

Stars 

13 

9% 

Celestial Motion & Navigation 

8 

6% 

Solar System 

7 

5% 

Cosmology 

6 

4% 

Scale & Structure 

6 

4% 

Atoms & Light 

4 

3% 

Gravity 

3 

2% 

Music & Art 

2 

1% 

Night sky & Constellations 

2 

1% 

Telescopes & Optics 

2 

1% 

Astrohiology & Exoplanets 

2 

1% 


Given that other portions of the categorization scheme currently demonstrate trends over time, the category related to 
content was also analyzed for trends over time. While “General Knowledge” is most studied content area over all three 
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research eras, it has decreased in prevalence. In the 1927-1980 period, “General Knowledge” accounted for 60% 
(n=15) of all categorized studies, decreasing to 40% (n=17) in the period between 1980 and 2000, and to 26% in the 
period since 2000 (n=20). While “Sun-Earth-Moon” has remained relatively stable in frequency, other topics appear 
in good numbers in the period since 2000, including cosmology, scale and structure, and Solar System. Nonetheless, 
one would not expect “General Knowledge” to disappear, as it is the content area most often engaged when studies 
focus on constructs such as “Nature of Science,” “Scientific Inquiry,” “Teaching Strategies,” and “Spatial Reasoning.” 
As researchers more aggressively venture into these domains, the data might show an increase in the number of 
“General Knowledge” content studies. 


DISCUSSION 

In this first light analysis paper, we found that there have been at least three times as many dissertations and graduate 
theses produced in AER than expected, and that these works were being completed far earlier in AER’s history than 
we imagined. These scholarly works display an extraordinary range of theoretical frameworks and methods, as 
demonstrated statistically. By and large, we judge the majority of these works to be rigorous and robust; in some cases, 
we judge the works to be quite brilliant and insightful, although we had never heard of the papers or their authors 
previously. Further, most AER topics one might wish to investigate have been touched on by one of these largely 
unknown scholarly treasures. Indeed, we found that many of our own ideas related to research questions for future 
projects had already been thought of before, by researchers perhaps decades ago. This certainly does not mean that 
we should not pursue answers to our questions; rather, if our field is to make progress, we must pursue those questions, 
building upon the findings of previous studies. 

These works have much to teach us, yet prior to insertion in the /STAR repository most of these studies were not 
previously available in digital format, or in many instances, outside of the walls of their home libraries. We can only 
wonder how many more useful existing works are hidden in the shelves of libraries across the planet, or in the pages 
of journals unknown to us. If this sample reflects the greater problem of dispersed research, it is safe to assert that the 
number of “hidden works” is quite large. 

We must collect these works. Until the field can accomplish that task, there is no process by which we can organize 
and begin to digest the whole of what we know about human thinking as it relates to astronomy. Without such a 
comprehensive organization of knowledge, it would be reasonable to say that astronomy education research has been 
a “pre-watershed” endeavor, with activity tending to be disparate, and perhaps a bit haphazard. According to Illich 
(1973), professional endeavors experience watershed moments when practitioners begin to ask questions which 
require them to reconsider their beliefs, seek out novel sources of information, and reorganize the collective knowledge 
into new coordinating theories 

The international STudies of Astronomy education Research (/STAR) database provides a reasonable pathway in 
which scholars may come to know, value, and build upon the work of a larger base of researchers, from a variety of 
countries, in order to support wider AER research collaborations. By building a database in which scholarly works 
from the larger international community can be highlighted, /STAR seeks to bring a more equitable balance to the 
recognition and subsequent influence of works from a broader international base of researchers while fostering a more 
collaborative international research environment. Such a watershed moment in AER would have the potential to 
facilitate collaborations and to enhance their effectiveness. Research may become more rigorous based upon an 
expanded library of tested research approaches. Research may also become more meaningful, drawing upon a broader 
conception of the ways that a wider diversity of humans can interact with the content and processes of astronomy. The 
/STAR online repository, with its growing and evolving representation of the various products of research enterprises 
around the world, is one pathway toward that first watershed in AER. 

With the /STAR repository as a structure upon which to move forward, we call for a new level of international and 
cross-disciplinary collaboration and coordination to better target key questions within the domain of AER. We 
encourage the submission of a wide variety of AER publications to the repository from all relevant parties; the use of 
/STAR as a means to organize our collective works; and the creation of vastly expanded reviews of the literature 
leading to new theoretical conceptions and methods in the field of astronomy education research. 
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